We study the problem of estimating a nonparametric probability distribution under a family of losses called Besov IPMs. This family is quite large, including, for example, L^p distances, total variation distance, and generalizations of both Wasserstein (earthmover's) and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide both lower and upper bounds, identifying precisely how the choice of loss function and assumptions on the data distribution interact to determine the mini-max optimal convergence rate. We also show that, in many cases, linear distribution estimates, such as the empirical distribution or kernel density estimator, cannot converge at the optimal rate. These bounds generalize, unify, or improve on several recent and classical results. Moreover, IPMs can be used to formalize a statistical model of generative adversarial networks (GANs). Thus, we show how our results imply bounds on the statistical error of a GAN, showing, for example, that, in many cases, GANs can strictly outperform the best linear estimator.

We study minimax convergence rates of nonparametric density estimation under the Huber contamination model, in which a ``contaminated'' proportion of the data comes from an unknown outlier distribution. We provide the first results for this problem under a large family of losses, called Besov integral probability metrics (IPMs), that include L^p, Wasserstein, Kolmogorov-Smirnov, Cramer-von Mises, and other commonly used metrics. Under a range of smoothness assumptions on the population and outlier distributions, we show that a re-scaled thresholding wavelet estimator converges at the minimax optimal rate under a wide variety of losses and also exhibits optimal dependence on the contamination proportion. We also provide a purely data-dependent extension of the estimator that adapts to both an unknown contamination proportion and the unknown smoothness of the true density. Finally, based on connections shown recently between density estimation under IPM losses and generative adversarial networks (GANs), we show that certain GAN architectures are robustly minimax optimal.

As shown in several recent papers [Liu et al., 2017, Liang, 2018, Singh et al., 2018, Uppal Namely, the estimators used in past work rely on the unrealistic assumption that the practitioner knows the Besov space in which the true density lies. The rest of this paper is organized as follows.

Neural Information Processing Systems http://nips.cc/

Its structure and organization could be substantially improved. The notation is unclear, and the terminology is not defined. For example, see lines 45-48. The formal problem statement (section 2.2) is vague, as well. Many technical terms are used without any context; they are not explained and, further, it is not clear how those concepts help the claims made in the paper.

The reviewers agree that this will make a good contribution to NeurIPS. Please read the reviewer suggestions and try to incorporate them into the final submission.

Additional Feedback: Overall, the paper addresses a very important issue into density estimation when contaminated by random outliers which is encountered in many machine learning problems. The theoretical guarantees for the linear and non-linear convergences rate and minmax bound is extremely useful for designing robust machine learning models. Unfortunately, I do not have the technical expertise to comment on the correctness of this approach. But for an out-of-area reviewer, I can see that this paper is well motivated and is written and structured well. As mentioned in my original review, experimental results with synthetic data could strengthen the paper and improve understanding of the paper, but it is not critical. The authors have explained using examples for potential applications for the theoretical results in Section 4.3 which seems good enough for me.

The reviewers agree that this paper would make a worthy contribution to NeurIPS. Please see the reviews for ways to improve the paper (especially regarding clarity and real world data). Experimental results with synthetic data could strengthen the paper but are not critical, if you think they could improve understanding of the paper, you might want to include it in the supplementary material.

We study the problem of estimating a nonparametric probability distribution under a family of losses called Besov IPMs. This family is quite large, including, for example, L p distances, total variation distance, and generalizations of both Wasserstein (earthmover's) and Kolmogorov-Smirnov distances. For a wide variety of settings, we provide both lower and upper bounds, identifying precisely how the choice of loss function and assumptions on the data distribution interact to determine the mini-max optimal convergence rate. We also show that, in many cases, linear distribution estimates, such as the empirical distribution or kernel density estimator, cannot converge at the optimal rate. These bounds generalize, unify, or improve on several recent and classical results.

We study minimax convergence rates of nonparametric density estimation under the Huber contamination model, in which a contaminated'' proportion of the data comes from an unknown outlier distribution. We provide the first results for this problem under a large family of losses, called Besov integral probability metrics (IPMs), that include L p, Wasserstein, Kolmogorov-Smirnov, Cramer-von Mises, and other commonly used metrics. Under a range of smoothness assumptions on the population and outlier distributions, we show that a re-scaled thresholding wavelet estimator converges at the minimax optimal rate under a wide variety of losses and also exhibits optimal dependence on the contamination proportion. We also provide a purely data-dependent extension of the estimator that adapts to both an unknown contamination proportion and the unknown smoothness of the true density. Finally, based on connections shown recently between density estimation under IPM losses and generative adversarial networks (GANs), we show that certain GAN architectures are robustly minimax optimal.